Fluent speech prosody: Framework and modeling

The prosody of fluent connected speech is much more complicated than concatenating individual sentence intonations into strings. Prosody framework and modeling should base on more understanding of both the production and perception of fluent speech. We analyzed speech corpora of read Mandarin Chinese discourses from a top-down perspective on perceived units and boundaries, and consistently identified speech paragraphs of multiple phrases that reflected discourse effect in fluent speech. Subsequent cross-speaker and cross-speaking-rate acoustic analyses of identified speech paragraphs revealed systematic cross-phrase patterns in every acoustic parameter, namely, F0 contours, duration adjustment, intensity patterns, and in addition, boundary breaks. We therefore argue for a higher prosodic node that governs, constrains, and groups phrases to derive speech paragraphs and show how to account for the tune and rhythm characteristic to fluent speech prosody through cross-phrase specifications. A hierarchical multi-phrase framework is constructed to account for the governing effects, with complimentary perceptual evidence. The framework specifies phrasal intonations as subjacent sister constituent subject to higher specifications; while output fluent speech prosody is cumulative results of contributions from every prosodic layer. To test our framework, we further construct a modular prosody model of multiple phrase grouping with 4 corresponding acoustic modules and have begun testing the model with speech synthesis. Finally, we argue that development of unlimited TTS could benefit most appreciably by capturing cross-phrase relationships in prosody modeling.